Amiga Collections: Amiga Amateur Radio User Group/AARUG UK #17 (199x)(Amiga Amateur Radio User Group UK)(PD)[WB][G4DCV].zip/AARUG UK #17 (199x)(Amiga Amateur Radio User Group UK)(PD)[WB][G4DCV].adf/IP.NetROM

Text File | 1989-11-30 | 34KB | 692 lines

[The following paper has been submitted to the 7th ARRL Computer
 Networking Conference.  You may read it and pass it on to others
 to read, but please do not reproduce it in any other publication.
 Copyright 1988 by Daniel M. Frank.]

[This is a DOS text version of a document originally produced
 with a word processing program.  In the translation process,
 some text formatting may have been disturbed, and the bold
 facing has definitely been lost.  So, it may not scan quite
 like the original.  Sorry.]


       TRANSMISSION OF IP DATAGRAMS OVER NET/ROM NETWORKS

                      Daniel M. Frank, W9NK
                        1802 Keyes Avenue
                     Madison, WI 53711-2006


ABSTRACT

One of the main design goals of the Internet Protocol was that IP
datagrams could be carried over existing local- and wide-area
networks.  This characteristic of IP makes it possible to build so
called "internetworks" out of existing network facilities.  We
built support for an existing Amateur wide area network, NET/ROM,
into the KA9Q TCP/IP package, allowing the use of NET/ROM to carry
IP datagrams, and adding features which make the KA9Q software
useful as a full duplex NET/ROM packet switch.  We have also shown
that NET/ROM may be used as a datagram network only, independent
of its transport and application layer facilities.


INTRODUCTION

In the late Seventies and early Eighties, the world of computer
communications consisted of many isolated local- and wide-area
networks.  Enough communications capacity existed to link the
entire country, and much of the world, into a single large network,
but the existing facilities were physically and logically
dissimilar.  They could not simply be "plugged together" to make
this large "internetwork" (or "Internet").

The designers of the Internet Protocol (the "IP" in "TCP/IP") were
committed to overcoming the obstacles that prevented an Internet
from developing.  They came up with two key ideas:

    o    Gateways can be established between networks.  A gateway
          is a computer which possesses the physical resources and
          software to connect with and speak to more than one kind
          of network.

    o    A single protocol can be developed whose messages
          ("datagrams") can pass through any network, hidden inside
          that network's "native" messages.  When a message with
          a datagram inside it encounters a gateway, the gateway
          "unwraps" the datagram, rewraps it in the native message
          of a second network, and sends it on its way.  If a
          datagram is too large to fit inside the native message
          type of a network, the gateway breaks it into pieces
          ("fragments"), each of which is then wrapped in a native
          message and sent on.

By using IP and gateways, the designers of the Internet have
created a global "network of networks", which today encompasses
hundreds of thousands of computer systems, connected to every
conceivable kind of network.

Amateur packet radio networking, like computer networking, consists
of many different network technologies and protocols.  Local AX.25
communications, digipeating, TexNet, ROSE, and NET/ROM, to name
only a few, coexist or compete for dominance as the network
technology of choice.  This competition is healthy, and is in the
spirit of amateur radio experimentation.  Any attempt to establish
one network over another as the single "standard" is both pointless
and doomed to failure.  No single standard can ever be imposed on
radio amateurs any more than it could have been imposed on computer
networks, given the investments already made in equipment,
software, and education.

As the dissimilar amateur networks grow in size, they meet up with
each other.  Sometimes they coexist on the same channels.  But
without gateways and some kind of Internet Protocol, each network
is an island of communication, unable to send or receive data
beyond its own shores.

The work described in this paper is a first step towards true
Amateur Radio internetworking.  Using the KA9Q TCP/IP package as
a basis, we have built a software system which functions as a
gateway between local TCP/IP networks and the NET/ROM network.  It
allows IP datagrams to be forwarded automatically and transparently
across existing NET/ROM facilities.  In addition, as a full
implementation of NET/ROM layer 3, it is capable of functioning as
a NET/ROM relay node (as opposed to an AX.25 endpoint), and as a
full duplex NET/ROM packet switch.


IP OPERATIONS OVER STANDARD AX.25 CONNECTIONS

In order to properly understand how we have interfaced to the
NET/ROM network, we should first examine how "ordinary" TCP/IP
operations take place over AX.25.  This description follows the ISO
OSI Reference Model (RM), a seven-layer classification of network
facilities.

From the bottom up, the layers used in packet TCP/IP operation are:

    (1)    A physical layer, made up of the radios, antennas, and
          modems used to generate and carry the tones used to
          convey digital data from one place to another.

    (2)    A data link layer, made up of HDLC and AX.25, used to
          format and address the data, detect errors and discard
          bad packets.  The link layer only knows about and
          communicates with stations with which we are directly
          connected.  In the case of packet radio, this means
          stations with which we have reliable, direct
          communications.  (Digipeating doesn't count, for purposes
          of this discussion.)

    (3)    A network layer, responsible for routing packets to their
          destinations through one or more link-to-link hops.  The
          main distinction between the data link and network layers
          is that the network layer provides facilities for
          communication between stations not directly connected.
          The network layer has to have some concept of routing,
          that is, the path to be taken by a packet to reach its
          destination.  We use IP as our network protocol.

    (4)    A transport layer, responsible for reliable end-to-end
          communications.  Our network layer does not guarantee
          that a packet will actually reach its destination.  While
          AX.25 provides link layer acknowledgement and
          retransmission, it does not guard against nodes which go
          down, software errors, or a destination station which is
          not on the air.  The transport layer provides for an
          acknowledgment to be sent from the packet's ultimate
          destination, and for retries in case that acknowledgment
          doesn't arrive within a reasonable amount of time.

        The other function of a transport layer is multiplexing.
          The network layer provides only host-to-host addressing.
          However, a computer can have many users, and provide many
          different services.  The transport layer takes incoming
          packets from the network and directs them to the proper
          programs based on information contained in the transport
          header portion of the packets.

        Our transport protocol is TCP, the Transmission Control
          Protocol.

    (5)    The session layer is mainly involved with providing
          services to individual programs within the computer.  It
          is not of importance for the current discussion.

    (6)    The presentation layer is mainly concerned with the
          uniform formatting of data, or its conversion between
          different character sets.  Some of the TCP/IP user
          programs have a very simple presentation "layer" which
          maps plain text messages in the native character set of
          the user's computer, to and from ASCII with a standard
          line-ending convention.

    (7)    The application layer is made up of the various programs
          and services that use networking facilities.  Users of
          TCP/IP mainly make use of telnet, for keyboard to
          keyboard chat and remote login, smtp for automated
          transfer of mail, and ftp, for easy exchange of files.



LINK LAYER MULTIPLEXING

As can be seen from our description, local TCP/IP operation uses
regular AX.25 communications for its link layer.  An AX.25 packet
containing an IP datagram contains a special code in the protocol
ID (PID) field of its header.  This allows the link layer software
to forward the contents of the packet to the proper part of the
KA9Q package, in this case the IP routing code.

If the AX.25 packet contained a PID of "no level 3", the link layer
would forward it to a different part of the package, in this case
the AX.25 session code, which allows users of the package to hold
"regular" AX.25 conversations, bypassing all layers between the
link and application layers.  (This brings up an important point
about the reference model we've presented:  an implementation may
not contain certain layers from the RM if the services they would
have provided are unused or unneeded.)

This switching of packets at the link layer based on their PIDs is
known as link layer multiplexing.  Multiplexing at the link layer
is extremely useful, because it allows different network layer
protocols to share the same data link services, and often the same
link connections.  Link layer multiplexing is what allows the KA9Q
software with NET/ROM support to act as a digipeater, an IP relay,
and a NET/ROM relay node, all on the same channel, through the same
TNC.


AN OVERVIEW OF NET/ROM

Now that we understand how IP datagrams are carried over packet
radio links, we should examine how NET/ROM operates.  Again, we
will use the ISO OSI Reference Model as our framework:

    (1)    The physical layer is the same, i.e. radios, antennas,
          and modems.

    (2)    The data link layer is again AX.25, but the Protocol ID
          field of NET/ROM packets is set to a special NET/ROM ID.

    (3)    The network layer of NET/ROM handles the automatic
          routing of packets to their destination.  A NET/ROM
          network packet header contains the source and destination
          callsigns of the NET/ROM endpoints.  There is no
          information about the route the packet will travel to its
          destination.  Instead, every node maintains a routing
          table based on routing adjacencies:  it receives
          broadcasts from other nodes which say, essentially, "I
          am willing to take traffic for such-and-such a node."
          When a NET/ROM node receives a network packet, it
          examines its routing table to see if anyone is willing
          to pass it on toward its destination.  If so, it hands
          off the packet to the next station.  If not, it simply
          throws the packet away without comment.

        The type of network communications service (as opposed
          to the routing techniques) used in NET/ROM (and IP) is
          usually called an unreliable, connectionless datagram
          layer, and the network layer packets are generally called
          datagrams.  The service is unreliable, because it does
          not guarantee or confirm ultimate delivery.  It is
          connectionless, because no circuit is established over
          which datagrams will travel.  (This contrasts with some
          public data network protocols, where before data may be
          sent to a remote system, a fixed path to that system must
          be set up through the network, with resources
          preallocated at every intervening node.  Each approach
          has its advantages and adherents.)

    (4)    The transport layer of NET/ROM uses what is called a
          sequenced packet protocol.  Unlike TCP, which delivers
          an unsegmented stream of bytes to the receiver, and is
          free to pack as many or as few bytes into each message
          as it likes, the NET/ROM transport delivers a sequence
          of packets.  The amount of data in these packets is
          determined by the amount of data in the AX.25 packets the
          NET/ROM user presents for transmission.  NET/ROM is not
          free to combine packets together for greater efficiency,
          although it can fragment and reassemble packets which are
          too large to fit in one of its transport messages.

        The NET/ROM transport protocol provides end-to-end
          delivery and acknowledgement, as well as demultiplexing
          of arriving messages by circuit number.  A NET/ROM node
          can be handling traffic for more than one circuit, or
          connection, at a time, and it directs that traffic
          internally by examining the circuit number field of the
          transport header.

    (5)    The session layer is not present in NET/ROM.

    (6)    The presentation layer is not present in NET/ROM.

    (7)    The application layer is what a user sees when he or she
          connects to a NET/ROM node.  It is responsible for
          responding to user commands to list routes and nodes, and
          establish connections.  "No layer 3" AX.25 packets
          arriving at a NET/ROM node are shunted directly up to the
          application layer, while "NET/ROM" PID packets are
          forwarded up to the NET/ROM network layer.  This link
          layer multiplexing should be familiar from our earlier
          discussion.


A full explanation of how the NET/ROM software works is beyond the
scope of this paper.  The reader is referred to the NET/ROM manual
for further details.


NETWORK AND INTERNETWORK

Our presentation of the ISO OSI RM has been somewhat simplified.
In particular, the ISO recognizes a subdivision of Layer 3 into a
Network Layer (3A) and an Internetwork Layer (3B).  Strictly
speaking, the Internet Protocol (IP) is a 3B protocol, while the
NET/ROM network service is a 3A protocol.  To put it somewhat
crudely, IP is an Internetwork Layer because its messages can be
routed through multiple logically and physically distinct networks.
The same cannot be said of X.25, for example, or of NET/ROM layer
3.  Our NET/ROM support in the KA9Q package reflects this
distinction.

The KA9Q NET/ROM software is not a full NET/ROM implementation.
That was unnecessary for our purposes.  We didn't need the NET/ROM
transport protocol, since our reliable end-to-end services are
already provided by TCP.  We didn't need the application layer, for
similar reasons.  What we did need was an existing network service
that could carry our IP datagram traffic to remote destinations
simply and easily.  The NET/ROM network layer was sufficient for
this purpose.

We use the NET/ROM nodes as a datagram network.  When we have
traffic to pass through a local NET/ROM, our software makes sure
we have an AX.25 connection to that node, then puts a NET/ROM layer
3 header on our IP datagram and sends it off to the NET/ROM via an
AX.25 packet with a protocol ID of NET/ROM.  The NET/ROM link layer
sees the protocol ID and passes the packet to its network layer,
which examines its routing table and passes the packet on to the
appropriate neighboring NET/ROM.  This process continues until the
packet arrives at the destination computer running the KA9Q
software, where it is unwrapped and passed back up to the IP code.

At no point is the NET/ROM user interface or transport layer
involved.  We do not have to issue CONNECT commands, or make use
of NET/ROM virtual circuits in any way.  The NET/ROM nodes accept
and pass our datagrams because they do not examine the contents of
network datagrams not specifically addressed to them.  We are able
to take advantage of the link-layer acknowledgements and automatic
routing of the NET/ROM system without the overhead of its higher
level services.


SOFTWARE ARCHITECTURE

Let's examine how this is done in more detail.  Our once-simple
protocol stack has grown a bit by now.  Let's have a look at it:

    (1)    The physical layer is basically unchanged (although we
          did added another physical layer service, described
          later).

    (2)    At the data link layer, we still have AX.25.  However,
          the link layer now multiplexes three different kinds of
          packets.  "No level 3" packets still go up the AX.25
          session code, and IP packets will go directly up to layer
          3B, but now we also direct packets with a NET/ROM PID to
          the NET/ROM 3A routing layer.

    (3A)    Incoming packets with a NET/ROM PID go to the network
          layer.  This is a full implementation of NET/ROM layer
          3.  It has its own routing table, similar to that found
          in any NET/ROM node.  It sends NODES broadcasts, which
          update the routing tables of neighboring NET/ROM nodes,
          and updates its own routing table on receipt of NODES
          broadcasts from those neighbors.

        The NET/ROM layer examines incoming NET/ROM datagrams to
          see if our station is their destination.  If the
          datagrams are not for us, the routing table is examined
          to see if we can forward them on to a neighboring node
          for handling.  If we can, they are sent back down to the
          link layer to continue their journey.  In other words,
          a station running the KA9Q package with NET/ROM support
          can act as a NET/ROM relay station.  As far as
          neighboring NET/ROM nodes are concerned, they are simply
          passing traffic on through another NET/ROM.

        If a NET/ROM datagram is for us, the network layer makes
          sure that it isn't a NET/ROM transport packet.  If it is,
          it is dropped.  If it isn't, it is sent up to layer 3B.

    (3B)    The internetwork layer contains the IP router and
          protocol code.  (Remember that IP has its own routing
          table and algorithms!).  It receives AX.25 traffic with
          a protocol ID of IP, as well as IP datagrams arriving in
          NET/ROM network datagrams.


The remaining layers are the same as before, so we won't repeat
them.


IP ROUTING VIA NET/ROM

The IP routing table is similar, although not identical, to the one
used for NET/ROM.  It contains two kinds of entries, which we will
call local routes and gateway routes.

A local route consists of an IP address and an interface name.  The
KA9Q software supports multiple interfaces, similar to the way that
NET/ROM supports both a TNC's modem and its serial port.  One
component of a route, both in NET/ROM and the KA9Q package, is the
interface through which an outgoing datagram should pass.  (The
main difference is that, while NET/ROM only supports AX.25 and two
interfaces, the KA9Q code supports many different link layers and
an almost unlimited number of interfaces.)  When a local route is
found in the IP routing table, this means that the station with the
given IP address is on the local subnet, which for packet radio
purposes means that it is within radio communications range.  The
datagram is forwarded to the link layer with an indication that
direct delivery should be attempted.

A gateway route consists of an IP address, an interface name, and
a gateway IP address.  When we encounter a gateway route, it means
that the station in question is not on our local subnet (i.e. not
within radio range), and must be reached via a relay station, or
gateway.  (You will recall the idea of gateways from our earlier
introduction of internetworking.)  The IP datagram is forwarded to
the link layer with an indication that the message should be sent,
not directly to its destination, but to the gateway station, which
will make an attempt to reach the recipient, perhaps via another
gateway.

When we added the NET/ROM support, we were concerned that it be
fully transparent to the IP layer, both out of concern over proper
design, and out of a desire to avoid any unnecessary rewrite of the
existing code.  One key assumption in the KA9Q IP routing software
is that of routing adjacency:  the IP layer makes the assumption
that it can reach, via the interface given, some IP address
mentioned in the route entry (either the recipient or the gateway).
However, we are using NET/ROM precisely because there is no IP
station within radio range who can handle our traffic.  In order
to maintain the adjacency assumption at the IP layer, we had to
simulate the presence of an adjacent IP station in the NET/ROM
code.

An IP route which uses the NET/ROM support looks just like any
other routing table entry:  it consists of a destination, an
interface, and an optional gateway.  The only difference is that
the interface is called "Netrom", and it's not a link layer
interface at all, although it appears that way to the IP routing
code.  When the IP layer sends a datagram down to the NET/ROM
"interface" for handling, it is actually calling a small stub
routine above the NET/ROM routing code.  This stub looks up the IP
address in a table which associates IP addresses, used at the IP
layer and above, with AX.25 callsigns, used by NET/ROM's network
layer.  If it finds an entry for the given IP address, it creates
a NET/ROM network layer datagram header with a destination address
set to the AX.25 callsign found in the association table, prepends
this header to the IP datagram, and hands it off to the NET/ROM
routing code.

The NET/ROM routing software now handles the datagram exactly as
it would any NET/ROM traffic coming in from outside:  it checks to
see if there is an entry for the destination AX.25 callsign in its
routing table.  If there is, it opens a link layer (AX.25)
connection to the neighboring NET/ROM node advertising the best
quality route, and forwards the message into the NET/ROM network,
to be delivered (ultimately) to the station whose AX.25 address is
that in the destination field of the NET/ROM network header, and
whose IP address was that of the destination or gateway in the IP
routing table entry.

This approach was extremely successful.  Not one line of code
needed to be changed in the IP routing code of the original KA9Q
package.


FEATURES TO SUPPORT NET/ROM PACKET SWITCHING

As implied above, the KA9Q package allows an almost unlimited
number of interfaces to be used for receiving and forwarding
packets.  At the IP layer, datagrams are routed from interface to
interface using information from the IP routing table.  We added
a similar functionality to the NET/ROM routing layer, allowing it
to receive and send NET/ROM traffic on any AX.25 interface it is
configured to use.  This feature allows the KA9Q software with
NET/ROM support to be used as a multi-port full duplex NET/ROM
packet switch, using standard TNCs or modem boards available for
the IBM PC.  This is vastly superior to the practice of wiring
together several NET/ROM TNCs with a diode bridge.  There is no
possibility of collisions, since each TNC has its own serial port
or bus address, so the interfaces can all run at full duplex, full
speed through the switch.  In addition, this arrangement can be
used with the high-speed interfaces and modems now becoming
available, far exceeding the capabilities of a standard TNC.

In several places in the United States, excess bandwidth on
commercial data links is being used to carry NET/ROM traffic.
These "wormhole" links work fairly well when there is only one
NET/ROM at each end, but their performance degrades quickly if more
are added.  Beyond the difficulties inherent in the diode bridging
scheme shown in the NET/ROM manual, there is an additional problem
not amenable to a simple hardware solution.  Most of the data links
being used are running through time-domain or statistical
multiplexing hardware, or through public data networks.  While all
of these provide some kind of carrier detect indication, in almost
every case that indication comes far too late to avoid collisions.
Carrier sense simply doesn't work, since the carrier indication
isn't there when it is needed, and arrives just in time to cause
unnecessary delays afterwards.  Performance of such an arrangement
is likely to degrade to below that of schemes using no carrier
sense at all.

NET/ROM nodes use a simple serial framing method to communicate
with each other over their serial ports.  We have added support for
this framing method alongside the "KISS" protocol which the KA9Q
package normally uses to communicate with TNCs.  It is possible to
plug a number of NET/ROM TNCs directly into the serial ports of an
IBM PC, and use the PC as a switch.  Some of those NET/ROM TNCs can
be at the ends of "wormhole" links.  These links can run at full
duplex with no collisions, thus getting maximum performance and
almost zero retries (assuming reliable data lines and serial
interface hardware).  The NET/ROM serial interface code is
instrumented to provide statistics on traffic volume and error
rates on its serial ports.


LESSONS LEARNED

The creation of the NET/ROM code has provided some interesting
lessons on how we should and should not go about building amateur
packet networks.  One of these lessons became apparent before we
even thought of writing the NET/ROM code.

We would probably not even have added the NET/ROM layer three
support to the KA9Q package had there been an easier way to
accomplish what we wanted, which was to use NET/ROM networks to
handle our IP traffic until we could build our own IP network.
Unfortunately, the NET/ROM software has a rather unfriendly link
layer multiplexor.  It sends "no layer 3" packets to the
application layer, and "NET/ROM" packets to the NET/ROM network
layer, but anything else it consigns to oblivion.  We could have
built fairly simple code to establish connections and send our IP
traffic over NET/ROM transport circuits, but any packet with a
protocol ID of "IP" was simply dropped by the NET/ROM software.
So, lesson number one:

     If you're going to build a networking product, write the
     multiplexing code to be inclusive, rather than exclusive.
     In other words, if you get something with an unfamiliar
     protocol ID, wrap it up and send it on, remembering to
     regenerate the PID properly on the other end.


Another problem we encountered was the lack of a protocol ID field
in the NET/ROM network layer header.  Both AX.25 packet headers and
IP datagram headers contain a field which indicates what sort of
higher level protocol stuff is packaged inside.  This is not unlike
the cans on your grocer's shelf:  without a label, you have a hard
time telling the beets from the beans.  The AX.25 protocol ID field
makes link layer multiplexing possible, and the protocol ID of an
IP datagram header allows many higher level protocols to use its
internetworking services.  Because the NET/ROM network header does
not contain a protocol ID field, there is no straightforward way
to put anything but a NET/ROM transport packet inside.  This is
unwise.  The authors of NET/ROM may well have been unaware that
anyone would ever attempt a project such as ours, but by leaving
this feature out of their network layer, they made it difficult,
if not impossible, to ever introduce other transport protocols into
their product line.  So, lesson number two:

     Include a protocol ID field in your link and network
     layer headers, even if you can't think of a use for it
     yet.  Make it big enough to be useful, and offer to be
     the repository of assigned PIDs, so that a standard
     develops.


In experimenting with the auto-routing code in our NET/ROM network
implementation, we discovered something that is a common complaint
among NET/ROM operators.  This can be summed up by the dictum,
"Just because you can hear them, doesn't mean they can hear you."
It is not unusual to have neighboring nodes that are "alligators"
(big mouth, tiny ears) or for your node to be a "rabbit" (big ears,
tiny mouth).  Also, band openings on two meters happen quite often,
and usually last just long enough for you to receive a routing
broadcast from a station from whom you will never hear again - at
least until the next band opening.  Either situation leaves your
routing table cluttered with impossible routes, which can lead to
repeated link-layer retries and transport layer failures.  Routes
based on band openings age out fairly quickly.  Ones based on deaf
neighbors come back, again and again.

After a bit of experience with this phenomenon, we added the
"nodefilter" feature to our implementation.  The user may specify
a list of nodes which are the only ones from which route broadcasts
will be accepted, or alternately, may specify a "reject list" of
nodes whose broadcasts will be routinely ignored.  The lesson:

     If your routing method involves broadcast in an
     asymmetrical or inconsistent communications environment,
     provide a way to restrict the routes accepted to those
     offered by reliable nodes.


The implementation described was actually the second one we did.
The first one grabbed AX.25 interfaces away from the IP part of the
KA9Q code, and could only be used for NET/ROM and regular AX.25
traffic.  This appeared to be a horrible idea from the moment the
first version was completed, and prompted an immediate rewrite,
producing a program that could act as a packet switch for IP as
well as NET/ROM.  The lesson here (besides "look before you leap")
is:

     If you're going to build a packet switch for amateur use,
     support link layer multiplexing, and try to make it
     multi-purpose.  This is a hobby, and the radios, tower
     space, and dollars are in short supply.  The more stuff
     you can do with a single one of each, the happier you
     will be in the long run.


EXPERIENCE

At this writing, experience with the software is necessarily
limited.  It is only now being made an official part of the
official KA9Q release (thanks, Phil!), so it has not been widely
available as for as long as we would have wished.  Still, it has
found its way into enough hands for us to have some preliminary
measurements and impressions.

The Madison, Wisconsin NET/ROM node (MAD) is connected to a node
(MQTA) in Marquette, in the Upper Peninsula of Michigan, via a
multiplexed commercial data line.  We have conducted tests between
W9NK, in Madison, and KV9P, in Alpha, Michigan.  Both stations sent
periodic routing broadcasts to announce their presence to their
local nodes.  The path chosen by NET/ROM was:

    W9NK <-> MAD <-> MQT <-> MQTA <-> IRN <-> KV9P

where MQTA and IRN were NET/ROM or TheNet nodes in Marquette and
Iron Mountain, Michigan, respectively.  All nodes except the two
using the data line were on two meters, with a speed of 1200 baud.

Performance was surprisingly good, with TCP round trip times
settling in around 12 to 20 seconds, with a standard deviation of
about nine seconds.  In spite of the number of hops, performance
was good enough to hold fairly coherent keyboard-to-keyboard
conversations.

We did note at least one case where duplicate copies of datagrams
were delivered by the NET/ROM network.  Since TCP discards
duplicates, this causes no problem in normal operations, but in the
case we noticed it resulted in two replies to the same ICMP Echo
Request message (produced by the ping command).

Feedback from other users, particularly N0AN in Iowa, illuminates
a serious problem with the management of existing NET/ROM networks:
the routing tables of these networks are so inaccurate that many
experienced users don't use the network layer facilities at all!
BBS mail forwarding scripts are set up to establish connections to
the local NET/ROM node, request a transport connection to a
selected neighbor, then from that neighbor to another, and so forth
to the NET/ROM node in their destination area.  They have
discovered that, without human intervention, many NET/ROM networks'
routing facilities break down and become unusable.

This problem has some impact on normal operations, in the sense
that these multiple transport sessions do not in any sense add up
to end-to-end protocol support.  There is in fact no transport
facility (as we understand the term) in use in these cases, since
no acknowledgements travel from one end of the communications path
to another.  There may as well be no transport layer in NET/ROM
under these circumstances; the overhead would at least be
substantially lower, with no additional loss of reliability.

Unfortunately, networks in such a pathological state are unusable
by our TCP/IP software.  Since we make no use of the NET/ROM
transport layer, we must rely entirely on the accuracy of the
network layer routing tables to support the forwarding of our
packets to their destinations.  If these tables are not correct,
our traffic will not get through.

The good news is that, in some areas where the NET/ROM operators
are also working with TCP/IP, this problem is forcing them to pay
attention to the quality of their routing tables.  As we have noted
above, NET/ROM is somewhat short on facilities to do this, but a
few things can be and are being done with the tools available.  One
side-effect of the TCP/IP NET/ROM support may be an improvement in
quality of service to all NET/ROM users!


FUTURE DEVELOPMENT

We hope, at some point, to produce a version of this support that
can be put into ROM and used in a dedicated packet switch for
hostile environments.  Such a switch would allow us to begin
building IP networks, while also offering superior performance to
the NET/ROM community.  It is our hope that the two user
communities can work together, sharing resources to build a better
network than either could alone.


ACKNOWLEDGEMENTS

Sincere thanks are due to Phil Karn, KA9Q, who would have been rich
(or at least, richer) by now if he hadn't been dedicated to
improving the state of the art for all radio amateurs.  Also,
thanks to everyone on the tcp-group mailing list, who contributed
helpful comments during the design and development of the NET/ROM
code.  Special thanks also to Howard Leadmon, WB3FFV, and John
Limpert, N3DMC, who got it to compile under Unix.

Individuals who helped test or provided feedback on experience with
the software were:  Duane Brummel, NX9K; Hasan Schiers, N0AN; and
Dave Reinhart, KV9P.

Phil Karn, KA9Q, reviewed the first draft of this paper, providing
many helpful comments and suggestions.